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(54) Title: PROGRESSIVE TRANSMISSION OF IMAGES 



(57) Abstract 

A method for the progressive 
transmission of images from a server to 
a workstation comprises encoding the 
image in multi-dimensional format, us- 
ing the orthogonal wavelet transform, 
and storing the encoded image as a dig- 
ital file on the server. A user request- 
ing the image at a workstation sees, 
first of all, a small thumbnail image. 
If further detail is required, a request 
is made to the server which sends a 
"detail" image consisting only of in- 
formation needed to upgrade the low- 
resolution image to a higher-resolution 
image. This approach saves transmit- 
ting across the network information that 
has already been sent in connection 
with the original low-resolution image. 
The system may be of particular use in 
the hospital environment, for the trans- 
mission of medical images. 
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PROGRESSIVE TRANSMISSION OF IMAGES 

The invention relates to progressive transmission cf 
images, and in particular although net exclusively to the 
oroaressive transmission cf medical images across a 
computer network from a server to a workstation. 

There is often a need, within the hospital 
. environment, for physicians and surgeons to have rapid 
access to a variety of medical images such as, for 
example, x-ray images or CAT scan images. Typically, 
the images may be stored in digital form on a central 
server so that they may be called up as required by the 
user on a local workstation or display client. 

The problem with such an arrangement is that the 
transmission cf images, particularly detailed medical 
images, across a network puts a great strain on the 
available bandwidth. In a realistic hospital 

environment, the time taken for transmission of one 
detailed image is likely to b-> high because of the 
competition for bandwidth arising from other users of the 
system who may be attempting to view different images at 
the same time on other workstations. A further 
difficulty is that the user may wish to see only one cf 
a series of perhaps rather similar images, and it is cr 
25 course extremely inefficient to have to transmit each of 

those images, one by one, in full detail before the user 
is able to choose the image he or she actually requires. 

One method of dealing with this problem is to 
present the user with a set of small "thumbnail" images, 
3C each of which is a low- resolution version cf one cf the 

images cf the set the user is interested in. The user 
selects the required image using the displayed thumbnail 
images, the required image then being sent in higher 
resolution form from the server to the workstation. 



20 
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There are two ways in which the image information 
may be updated from a low- resolution thumbnail image to 
a high- resolution display of the same image. The first 
way is to retransmit the full concent of the high- 
resolution image. This represents a wastage of 
bandwidth, since some of the image information is already 
present at the viewing console. The second way in which 
the display may be updated is to transmit only the 
difference between the current image resolution and the 
desired high-resolution image from the server to the 
workstation. In the case of a wide area distributed 
viewing system, the retransmission of full resolution 
images represents a potential problem: for an individual 
image the retransmission of the full resolution image may 
not be significant; for the case in which there are many 
active viewing consoles, and a steady traffic of viewing 
and selection operations, the retransmission of the 
entire image may represent a significant bandwidth 
reouirement over the transmission of the detail only. 
0 It is an object of the present invention at least to 
alleviate the problems of the prior art. 

It is a further object to provide an efficient 
progressive transmission system having an improved 
(smaller) bandwidth requirement. 
5 It is a further object to provide a system in which 

progressive transmission may be applied to any selected 
portion of an image that the user wishes to see at 
higher- resolution . 

According to a first aspect of the present invention 
there is provided a method of progressively transmitting 
images across a computer network from a server to a 
workstation, the method comprising: 

(a) storing an image on the server as a digital 
file, the file comprising a low-resciuticn portion 



BAD ORIGINAL 



WO 96/29818 PCT/GB96/00623 



representative of a low- resolution version of the 
image, and a detail portion representative cf detail 
in the image not contained within the low-resolution 
version; 

(b) transmitting first information from the low- 
resolution portion to the workstation, and 
displaying the low-resolution version of the image 
at the workstation; 

(c) selecting from the displayed image a user- 
defined area to be viewed at higher resolution; 

(d) transmitting from the server second information 
from that part of the detail portion of the file 
which relates to the user-defined area; 

(e) combining at the workstation the second 
information and that part of the first information 
which relates to the user-defined area to create a 
higher-resolution version cf the user-defined area; 
and 

(f) displaying at the workstation the higher- 
resolution version of the user-defined area. 
The method of the present invention is particularly 

although not exclusively of use in the hospital 
environment, for the transmission of medical images such 
as echocardiographic images. There is often a 

25 requirements in a hospital environment to display a 

selection cf mosaic of images, which may typically be 
image slices of a 3D data set, or frames of an image 
sequence. The present invention allows the individual 
images making up the mosaic to comprise small low- 
30 resolution "thumbnail" images. A user who wishes to see 

r-.ore detail for one particular image, simply selects the 
image (for example by clicking on it using the mouse) . 
Further detail is then sent from the server, this further 
detail being combined with the information already held 
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at the workstation to produce a slightly higher 
resolution image. Yet further information may be 
requested from the server if the user needs to see an 
even higher resolution version of the image. In each 
case, the only information that needs to be transmitted 
across the network is the detail information, that is the 
difference between the current low-resolution version of 
zhe image already held at the workstation, and the 
required higher-resolution version. 

In some embodiments, the information may be sent 
automatically from the server. In other embodiments, a 
specific request may be sent by the workstation to the 
server each time further information is required. This 
request may conveniently be user-generated. 

The preferred encoding method is to use the 
orthogonal wavelet transform. This is a transform which 
is discrete in both the time domain and in the scale - 
space (wavelet) domain. The small extent of the 
functions in the spatial domain allows for rapid 
0 convolution. The orthogonal nature of the functions 

used in the preferred embodiment guarantees that the 
minimum amount of information needs to be sent across the 
network . 

The user may select a desired area of the image 
5 which he or she wishes to see in more detail. A desired 

region could for example be picked out on the screen with 
a mouse, preferably by dragging a box over the area of 
interest. If only an area of the image is selected by 
the user, the server sends information only on that 
selected area across the network. 

The invention extends to a computer network and/or 
system for operating the method of the present invention. 

The computer network may comprise a local area 
network or a wide area network, connected by wires or by 
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a wire-less link. The term "network" includes any means 
of remote communication between computers, and 
accordingly encompasses (without limitation) 
communication via the telephone system and/or satellites. 
5 Such means cf remote connection may be useful where for 

example the server is in a different hospital, and 
perhaps even in a different country, from the 
workstation . 

The invention may be carried into practice in a 
10 number of ways and one specific embodiment will now be 

described, by way of example, with reference to the 
accompanying drawings, in which: 

Figure 1 is a block diagram showing, schematically, 
the operation of the discrete wavelet transform; 
15 Figure 2(a) shows the discrete wavelet transform 

decomposition process; 

Figure 2 (b) shows the decimation process at 
successive scales; 

Figure 3 shows the Daubechies 4, 12 and 2 0 scaling 
20 functions, evaluated at different respective scales; 

Figure 4 illustrates the decimation process in 
graphical form; and 

Figure 5 shows how the mult i -resolution image may be 
stored . 

25 The preferred embodiment of the present invention is 

a method and/or system for the progressive transmission 
of medical images from a remote file server to a local 
workstation. Each image is stored cn the server in a 
multi-resolution format comprising a low-resolution 

30 representation which may be called up as a small 

"thumbnail" image cn the workstation, and a plurality cf 
"detail" representations. If the user wishes see the 
image at lowest resolution, the thumbnail image alone is 
sent. If, having viewed the thumbnail image, the user 
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wishes to receive a higher- resolution representation, the 
first "detail" representation is passed along the network 
co the workstation. At the workstation, the "detail" 
representation is added to the low- resolution image to 
• provide the required higher-resoluticr. image. If yet 

further resolution is required, the user can call upon 
the next "detail" representation, stored on the server, 
which in a like manner is added to the information which 
has previously been passed to the workstation to provide 
) an image at the next highest resolution. The process may 

be continued until the user has the image resolution he 
or she requires, or has an image of the maximum 
resolution which is available from the server. 

The first stage in the process is, of course, to 
5 take an original image and to encode it in digital form 

in a digital data file which forms the multi-resolution 
representation that is actually stored on the server. In 
order to split the original image up into a low- 
resolution image plus a plurality of "detail" 
0 representations or images, the preferred embodiment uses 

the orthogonal wavelet transform. 

The orthogonal wavelet transform is a discrete 
transform, similar to the Foulier transform but instead 
of using sin functions wavelet mother functions are used 
5 instead. Each wavelet mother function has a limited 

extent in wavelet space, which contrasts with the sin 
functions used for Foulier transforms which are of course 
infinite in extent. 

The wavelet transform has the following general 

0 form: 



W 



( r) V 



z-b) 
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The discrete wavelet transform, which is used here, 
is the same, except that the values of a and b are 
restricted to discrete values only. In the preferred 
5 embodiment, orthogonal wavelets are chosen, for example 

the Daubechies wavelets shown in Figure 3. 

For any given signal, one can compute a series cf 
weights for discrete values of a and b which, when 
multiplied by the basis function and scaled leads back to 
IC the representation of f . This can be done, in principle, 

without losses (other than those arising from rounding 
errors) . The fact that the wavelets are orthogonal 
guarantees the lowest possible error if certain terms are 
thrown away, for example if the user wishes to see an 
15 image of only intermediate resolution. 

The advantage of the present scheme is that the full 
set of coefficients or weights W(a,b) can be arranged 
into a multi-resolution representation of the original 
signal . 

20 As an illustration, consider the one-dimensional 

example shown in Figure 4. If one takes f (n) to be a 
discrete signal to be approximated, one can obtain a 
first approximation by calculating f ' where f ' (n) is 
calculated by determining the mean cf adjacent samples 

25 within f(n). For example, f ' (0) is half the sum of f(0) 

and f(l). Likewise, f'(l) is half the sum of f(2) and 
5(3). F' (n) may therefore be considered as an 
approximation g(n) to f(n). One may also define a 
"detail" representation h(n) as the difference between 

30 f (n) and g (n) . 

The process may be repeated, decimating the sample 
each time, thereby creating a plurality of individual 
"detail" representations, each indicative of the 
difference between the current approximation and the last 
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previous approximation. 

To store the image in multi-resolution format, the 
final approximated signal is kept, along with all of the 
detail signals. 

The process may be represented schematically at each 
stage by the block diagram shown in Figure 1. At each 
stage, an approximation signal g(n) and a detail signal 
h(n) is determined from the input signal. The 
approximation signal is then fed back into the output and 
a further approximation and a further detail signal 
obtained. At an appropriate stopping point, the final 
very low- resolution g(n) output is retained, along with 
ail of the detail outputs h(n). 

The structure shown in Figure 1 can be used for 
multi - resolution orthogonal biorthogonal , or quadrature 
mirror filters (QMF) where each of the above cases is 
specified by the chosen filter coefficients. 

The filters are simply denoted as low pass, g(n) and 
high pass, h(n), to present a general form, but in fact 
0 these filters are related to scaling function, <£(x), and 

wavelet function iMx) [2] . The decomposition process is 
illustrated in Figure 2. 

Each block has both a low pass filter and a high 
pass filter. The output of each filter is decimated by 
5 a factor of two. The resolution of the output cf the low 

pass filter is also changed because cf losing high 
frequency detail. 

The output of the filter chain, A : '"S, signal 
represents a discrete approximation of signal at the 
C resolution 2' m . The other output is called the detail 

signal, D ; " m S, at the resolution 2" T . If the original 
signal has N samples, then the total number cf samples ir. 
the detail and approximation signals is also N.. Thus the 
discrete wavelet representation has the same total number 
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of samples as the original signal. The smoothness of A2 : S 
depends on the shape of the scaling function. In general 
we would wish to have a smooth scaling function, but this 
function should satisfy some conditions such as 
orthogonality, support width etc. Dne cf the important 
things is the spatial support ..width) of the scaling 
function which divides the wavelet transform into compact 
into compact and non-compact wavelets. 

In compactly supported wavelets the scaling function 
and the wavelet function have mere regularity at the 
expense cf a large number of nor.- zero coefficients of 
h(n:-, which results in a large support for 0(x). This 
support width increases linearly with their regularity in 
some wavelets such as the so-called Daubechies wavelets 
15 (Daubechies, 1988) . As shown in Figure 3, increasing the 

number of coefficients of h(n), leads to an increase in 
the smoothness of the scaling function. Note that the 
three traces of Figure 3 have been evaluated at 
resolution in inverse proportion to their support, so 
20 that the x axis does not allow comparison of true 

support. Because of the compact support and 

orthogonality the Daubechies wavelets provide a suitable 
basis set. The Mallat basis set is constructed from 
solines in continuous space, then discretized, and hence 
25 is not compact. 

A multi-resolution decomposition is a method of 
representing signals at different scales of 
magnification. The concept cf multi-resolution 

decomposition was initially developed by Mallat [2] to 
3 0 address the problem of characterising image scenes in a 

scale independent manner. The approach cf Mallat is 
essentially a pyramidal decompositicn, along the lines cf 
Burt and Adelson, but uses values of h!n) and gin) such 
that the decomposition is onto an orthogonal basis. The 
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basic idea is that of successive approximation, together 
with adding detail signal from one approximation to the 
next . Assume that we have a ladder of spaces such that 

... c V z ~ V. c V. ~ V.. c V_., ... 

5 With the property that if 

f (x) € V. 

then 

f ■; x - 2 -' k ) e V . ; ke Z ; and f[2x) e v. . . 

10 If W. is the orthogonal complement of V. in V^; so we 

have 

v i - : = v i ® W : 

Thus, W : contains the detail signal necessary to go 
15 from V. to V.... If we have an approximation of a signal 

at a resolution corresponding to V., then a better 
approximation to the signal is obtained by adding the 
detail signal corresponding W. . This detail signal is the 
projection of the signal in W : . 
20 By interpolating the output 

A.-.S 

of the decomposition depleted in Figure (2; , one obtains 
an approximation 
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S (a) 



cf the signal Sin) at a resolution 2 : . The interpolation 
is defined by the following equation: 

. S ! .r.) = £ <\> (.-:-.<) A:-. S(k) 



(1) 



wnere 



10 denotes insertion of a zero between each sample of the 

discrete signal 



A 2 -.S 



This interpolation is a convolution between the 
.5 scaling function <t> and 



a:.-. S 



This particular interpolation represents an example 
in going from a representation cf the signal at one scaie 
20 to an approximation of the signal at the scale 
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:o 



immediately above it. If we wish to go from a 
representation of the signal at, say, a scale of 2" : to an 
approximation of the signal at some scale 2' : "- , where 
1>1, it is necessary to dilate the scaling function 
appropriately. 

The appropriate dilation may be performed as 
follows: if the scaling function of Equation (1) is at 
the resolution Ki.e. 2" : ) , then it belongs to V- . 



Since 



., c V. 



one may write 

<t> {2- l n) = £ h(k)${r.-k) 

k 

(2) 

This equation may be generalised to relate the 
scaling function at a scale 2- to the scaling function at 
a resolution 2 : *'- for any j. Therefore, 



S (r>) = 52 * (2 : '-n-k) A. T . S(k) 



i.3) 



For the sake of simplicity, the above examples have 
assumed a one-dimensional signal rather than a two- 
dimensional image, which of course is what we are really 
interested in. In dealing with images rather than 
sianals, it is necessary to use the corresponding two- 
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dimensional discrete wavelet transform !DWT) . The two- 
dimensional transform may again be represented by a 
repeated function block where, instead of a pair of 
outputs, we have four outputs corresponding to the low- 
frequency image component and the r.ign- frequency 
vertical, horizontal and diagonal components. The latter 
three image components correspond to the detail signals. 

For details of the two-dimensional decomposition, 
refer to Mallat [1] . It is possible to generalise 
Eauation (3) to the two-dimensional case: 



5 \r..r.) = £ £ 4> (2--* 1 n-ic) <t> (2-""- ~-2) A : , S'.k.i) 

(4) 

due to the separability of the two-dimensional 

decomposition . 

Figure 5 shows, in schematic form, how the 
information may be stored in a data file on the server. 
The lowest resolution image that is expected to be 
required (A) is stored, along with the sequences of 
detail information D : , D, and D, . If, for example, the 
lowest resolution image A is an 8 x 8 image, one will 
automatically obtain from the algorithm three sets of 
detail signals each of which also has size 3x8. The 
next level up is 15 x 16, and so on. To produce an image 
of 16 x 16 resolution, the three detail signals 3 ; are 
added into the low-resolution image A. Similarly, to 
create an image of 32 x 32 resolution, the detail images 
2. are added onto the previously-created 16 x 15 image. 

It will of course be understood that it is not 
necessary for the data file to be stored exactly as it is 
shown in Figure 5. All that is required is that the low- 
resolution image can be selected from the file and sent 
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across the network to the workstation followed, if 
reauested by the user, by each of the detail images in 
crder . 

In looking at medical images, it is cften useful to 
be able to view a particular region of interest in 
rreacer detail. The infrastructure of orthogonal, 

rr.ultiresoiution image decomposition provides an efficient 
way cf doing this, in terms of bandwidth requirements. 
Consider a region of interest, such as a heart valve on 
a cardiac image. By selecting an approximate region of 
interest on a low resolution, interpolated image, the 
-resent infrastructure allows one to view that region at 
full resolution without needing to increase the 
resolution outside of the specified region. 

The user first selects an area of interest (for 
example using a mouse) , and that area is then redrawn on 
the screen in greater detail. For example, if the user 
wishes to view in more detail only the top left-hand 
corner of the thumbnail image A, he or she simply selects 
that area and a signal is sent to the server instructing 
it to send further detail appropriate to that area only. 
With the file structure shown in Figure 5, the server 
would send only the detail information contained within 
the top left-hand corners of each of the three D. images. 
If the user then wishes to view that enlarged image in 
vet more detail, the server would send only the top left- 
hand corners of the D : images. 

In order to refine the detail over a particular 
user-selected area of the image, we need to determine the 
rar.ae of influence of all samples (pixels) within the 
area we are interested in, over all scales of the 
decomposition process. This range of influence is 
oleariv a function of the width cf the filters used, that 
is the impulse response cf the filters used in the 
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decomposition. Since we are typically using filters of 
very small length, this range of influence is fairly 
localised throughout all space. However, proper 

reconstruction can be carried out only if we can 
5 correctly identify ail of the samples, at each scale, 

which influence the desired image within the selected 
area . 

A general implementation involves tracing all of the 
influenced children of a pixel throughout the various 
10 components of the image in the transform space. 

Although tedious, this is quite within the capabilities 
of the skilled man. If orthogonal filters are used, as 
they are in the preferred embodiment, then the range of 
influence simply becomes the direct children (4) of each 

15 pixel. 

This allows great savings in bandwidth. Consider 
the requirement to view a region of microcalcif ication in 
a digital x-ray mammogram at a resolution of 256x256 
pixels. Mammograms typically have full pixel dimensions 
20 of 4096x4096 pixels. In such a situation, immediate 

enhancement of the region requires all the coefficients 
to go in resolution from, say a 512x512 icon to a full 
resolution region is less than 1% of the total 
coefficients necessary to rebuild the entire image at 
25 4096x4096 resolution. 

The results of our study show that the 
reconstruction of an image from its full wavelet 
decomposition represents by selecting an appropriate set 
of scales represents a valid method of building a multi- 
30 resolution framework for progressive transmission of 

echocardiographic images. We have shown that not only 
does this framework support the progressive updating cf 
"thumbnail" images, but also that smooth interpolation cf 
these small images by an appropriately dilated scaling 
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function lead to low resolution images at magnifications 
(scales) comparable to the original image. Such blurred 
images may be used, for example, to select with precision 
a particular region of interest, which one wishes to view 

5 in greater detail (i.e. at full resolution). 

The clear advantage of adopting this approach to the 
clinical viewing of images in a networked client/server 
environment, such as a teleconferencing or remote 
diaanosis application, is that communications bandwidth 

C is used optimally. Where a large number of clients are 

involved, and commercial networks are employed, the 
savings can be significant. Finally, even on local area 
networks using ethernet technology, the cut in overheads 
can represent a significant saving in bandwidth. 

15 

References 

[1] Daubechies, I., "Orthonormal basis of compactly 
supported wavelets", Communications on Pure and Applied 

20 Mathematics, vol. 41, pp 909-996, 1988. 

[2] Mallat, S. ( "A Theory for multiresolution signal 
decomposition: The wavelet representation", IEEE 

Transactions on Pattern Analysis and Machine 
Intelligence, vol. 11, no. 7, pp 674-692, 1988. 

25 [3] Burt, P. and Adelson, E., "The Laplacian Pyramid as 

a compact image code", IEEE Transactions on 
Communications, vol. 31, no. 4, April 1383, pp 532-540. 



BAD ORIGINAL 



WO 96/29818 



17 



PCT/GB96/00623 



CLAIMS : 

1. A method of progressively transmitting an image 
across a computer network from a server to a workstation, 
5 zhe method comprising: 

iai storing an image on the server as a digital 
file, the file comprising a low-resolution portion 
representative of a low-resolution version of the 
image, and a detail portion representative of detail 
0 in the image not contained within the low- resolution 

version; 

(b) transmitting first information from the low- 
resolution portion to the workstation, and 
displaying the low-resolution version of the image 

5 at the workstation; 

(c) selecting from the displayed image a user- 
defined area to be viewed at higher resolution; 

(d) transmitting from the server second information 
from that part of the detail portion of the file 

20 which relates to the user-defined area; 

(e) combining at the workstation the second 
information and that part of the first information 
which relates to the user-defined area to create a 
higher-resolution version of the user-defined area; 

25 and 

(f) displaying at the workstation the higher- 
resolution version of the user-defined area. 

2. A method as claimed in Claim 1 in which the detail 
20 portion of the file comprises a plurality of detail sub- 
portions, successive sub-portions containing information 
or. successively finer detail within the image. 

3. A method as claimed in Claim 2 in which third 
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information from a first sub-portion is transmitted from 
the server, the third information being combined at the 
workstation with the first information to create a first 
intermediate resolution version cf the image. 

4 . a method as claimed in Claim 3 in which fourth 
information from a second sub-portion is transmitted from 
the server, the fourth information being combined at the 
workstation with the first intermediate resolution 
version of the image to produce a second intermediate 
resolution version, of higher resolution than the first 
intermediate resolution version. 

5. A method as claimed in Claim 4 in which fifth 
information from a third sub-portion is transmitted from 
the server, the fifth information being combined at the 
workstation with the second intermediate resolution 
version of the image to produce a third intermediate 
resolution version, of higher resolution than the second 

0 intermediate resolution version. 

6 . A method as claimed in any one of the preceding 
claims in which the digital file is encoded using a 
transform having a basis function cf finite extent. 

7. A method as claimed in any one of Claims 1 to 5 in 
which the digital file is encoded using the wavelet 
transform . 



8 . A method as claimed in any one of Claims 1 to o in 
which the file is encoded using the orthogonal wavelet 
transform. 



5. A method as claimed in Claim 9 in which the 
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orthogonal wavelet transform uses a Daubechies' wavelet. 

1C. A method as claimed ir. any one cf the preceding 
riaims ir. which the combining at the workstation includes 
interpolation . 

11. A method as claimed in any one of the preceding 
claims including storing a plurality of images on the 
server, and displaying on the workstation a corresponding 
plurality of low-resolution versions of the images for a 
user to select . 

12. A method as claimed in any one cf the preceding 
claims in which information from the detail portion is 
transmitted from the server on receipt of a request from 
the workstation. 

13. A method as claimed in Claim 12 in which the request 
from the workstation is user- selectable . 
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